<?xml version="1.0"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Getting Connected with Endpoints</title>
</head>

<body>
<h1>Getting Connected with Endpoints</h1>

<h2>Introduction</h2>

<p>On a network, one can think of any given connection as a long wire,
stretched between two points. Lots of stuff can happen along the length of
that wire - routers, switches, network address translation, and so on, but
that is usually invisible to the application passing data across it.
Twisted strives to make the nature of the "wire" as transparent as
possible, with highly abstract interfaces for passing and receiving data,
such as <code class="API" base="twisted.internet.interfaces">ITransport</code>
and <code class="API" base="twisted.internet.interfaces">IProtocol</code>.</p>

<p>However, the application can't be completely ignorant of the wire.
In particular, it must do something to <em>start</em> the connection, and
to do so, it must identify the <em>end points</em> of the wire. There are
different names for the roles of each end point - "initiator" and
"responder", "connector" and "listener", or "client" and "server" - but the
common theme is that one side of the connection waits around for someone to
connect to it, and the other side does the connecting.</p>

<p>In Twisted 10.1, several new interfaces were introduced to describe
each of these roles for stream-oriented connections: <code class="API"
  base="twisted.internet.interfaces">IStreamServerEndpoint</code> and <code
  class="API" base="twisted.internet.interfaces">IStreamClientEndpoint</code>.
The word "stream", in this case, refers to endpoints which treat a
connection as a continuous stream of bytes, rather than a sequence of
discrete datagrams: TCP is a "stream" protocol whereas UDP is a "datagram"
protocol.</p>

<h2>Constructing and Using Endpoints</h2>

<p>In both <a href="servers.xhtml">Writing Servers</a> and <a
  href="clients.xhtml">Writing Clients</a>, we covered basic usage of
endpoints; you construct an appropriate type of server or client endpoint,
and then call <code>listen</code> (for servers) or <code>connect</code>
(for clients).</p>

<p>In both of those tutorials, we constructed specific types of
endpoints directly. However, in most programs, you will want to allow the
user to specify where to listen or connect, in a way which will allow the
user to request different strategies, without having to adjust your
program. In order to allow this, you should use <code class="API"
  base="twisted.internet.endpoints">clientFromString</code> or <code
  class="API" base="twisted.internet.endpoints">serverFromString</code>.</p>

<h3>There's Not Much To It</h3>

<p>Each type of endpoint is just an interface with a single method that
takes an argument. <code>serverEndpoint.listen(factory)</code> will start
listening on that endpoint with your protocol factory, and
<code>clientEndpoint.connect(factory)</code> will start a single connection
attempt. Each of these APIs returns a value, though, which can be important.
</p>

<p>However, if you are not already, you <em>should</em> be very
familiar with <a href="defer.xhtml">Deferreds</a>, as they are returned by
both <code>connect</code> and <code>listen</code> methods, to indicate when
the connection has connected or the listening port is up and running.</p>

<h3>Servers and Stopping</h3>

<p><code class="API" base="twisted.internet.interfaces">IStreamServerEndpoint.listen</code>
returns a <code class="API" base="twisted.internet.defer">Deferred</code>
that fires with an <code class="API" base="twisted.internet.interfaces">IListeningPort</code>.
Note that this deferred may errback. The most common cause of such an error
would be that another program is already using the requested port number,
but the exact cause may vary depending on what type of endpoint you are
listening on. If you receive such an error, it means that your application
is not actually listening, and will not receive any incoming connections.
It's important to somehow alert an administrator of your server, in this
case, especially if you only have one listening port!</p>

<p>Note also that once this has succeeded, it will continue listening
forever. If you need to <em>stop</em> listening for some reason, in
response to anything other than a full server shutdown (<code>reactor.stop</code>
and / or <code>twistd</code> will usually handle that case for you), make
sure you keep a reference around to that listening port object so you can
call <code class="API" base="twisted.internet.interfaces">IListeningPort.stopListening</code>
on it. Finally, keep in mind that <code>stopListening</code> itself returns
a <code>Deferred</code>, and the port may not have fully stopped listening
until that <code>Deferred</code> has fired.</p>

<p>Most server applications will not need to worry about these details.
One example of a case where you would need to be concerned with all of
these events would be an implementation of a protocol like non-<code>PASV</code>
FTP, where new listening ports need to be bound for the lifetime of a
particular action, then disposed of.</p>

<h3>Clients and Cancelling</h3>

<p><code class="API" base="twisted.internet.interfaces">IStreamClientEndpoint.connect</code>
will connect your protocol factory to a new outgoing connection attempt. It
returns a <code>Deferred</code> which fires with the <code>IProtocol</code>
returned from the factory's <code>buildProtocol</code> method.</p>

<p>Connection attempts may fail, and so that <code class="API"
base="twisted.internet.defer">Deferred</code> may also errback. If it does so,
you will have to try again; your protocol won't be constructed, and no further
attempts will be made.</p>

<p>Connection attempts may also take a long time, and your users may
become bored and wander off. If this happens, and your code decides, for
whatever reason, that you've been waiting for the connection too long, you
can call <code class="API" base="twisted.internet.defer">Deferred.cancel</code>
on the <code>Deferred</code> returned from <code class="API"
base="twisted.internet.interfaces.IStreamClientEndpoint">connect</code>, and the
underlying machinery should give up on the connection. This should cause the
<code>Deferred</code> to errback, usually with <code class="API"
base="twisted.internet.defer">CancelledError</code>; although you should
consult the documentation for your particular endpoint type to see if it may do
something different.</p>

<p>Although some endpoint types may imply a built-in timeout, the
interface does not guarantee one. If you don't have any way for the
application to cancel a wayward connection attempt, the attempt may just
keep waiting forever.  For example, a very simple 30-second timeout could be
implemented like this:
<pre class="python">
attempt = myEndpoint.connect(myFactory)
reactor.callLater(30, attempt.cancel)
</pre>
</p>

<h2>Maximizing the Return on your Endpoint Investment</h2>

<p>Directly constructing an endpoint in your application is rarely the
best option, because it ties your application to a particular type of
transport. The strength of the endpoints API is in separating the
construction of the endpoint (figuring out where to connect or listen) and
its activation (actually connecting or listening).</p>

<p>If you are implementing a library that needs to listen for
connections or make outgoing connections, when possible, you should write
your code to accept client and server endpoints as parameters to functions
or to your objects' constructors. That way, application code that calls
your library can provide whatever endpoints are appropriate.</p>

<p>If you are writing an application and you need to construct
endpoints yourself, you can allow users to specify arbitrary endpoints
described by a string using the <code class="API"
  base="twisted.internet.endpoints">clientFromString</code> and <code
  class="API" base="twisted.internet.endpoints">serverFromString</code>
APIs. Since these APIs just take a string, they provide flexibility: if
Twisted adds support for new types of endpoints (for example, IPv6
endpoints, or WebSocket endpoints), your application will automatically be
able to take advantage of them with no changes to its code.</p>

<h3>Endpoints Aren't Always the Answer</h3>

<p>For many use-cases, especially the common case of a <code>twistd</code>
plugin which runs a long-running server that just binds a simple port, you
might not want to use the endpoints APIs directly. Instead, you may want to
construct an <code class="API" base="twisted.application.service"
>IService</code>, using <code class="API" base="twisted.application"
>strports.service</code> , which will fit
neatly into the required structure of <a href="plugin.xhtml">the twistd
plugin API</a> . This doesn't give your application much control - the port
starts listening at startup and stops listening at shutdown - but it does
provide the same flexibility in terms of what type of server endpoint your
application will support.</p>

<p>It is, however, almost always preferable to use an endpoint rather
than calling a lower-level APIs like <code class="API"
base="twisted.internet.interfaces.IReactorTCP" >connectTCP</code>, <code
class="API" base="twisted.internet.interfaces.IReactorTCP">listenTCP</code>,
etc, directly. By accepting an arbitrary endpoint rather than requiring a
specific reactor interface, you leave your application open to lots of
interesting transport-layer extensibility for the future.</p>

<h2>Endpoint Types Included With Twisted</h2>

<p>The parser used by <code>clientFromString</code> and
<code>serverFromString</code> is extensible via third-party plugins, so the
endpoints available on your system depend on what packages you have installed.
However, Twisted itself includes a set of basic endpoints that will always be
available.</p>

<h3>Clients</h3>

<ul>
  <li>TCP.  Supported arguments: host, port, timeout.  timeout is optional.  For
  example, <code>tcp:host=twistedmatrix.com:port=80:timeout=15</code>.
  </li>
  <li>SSL.  All TCP arguments are supported, plus: certKey, privateKey,
  caCertsDir.  certKey (optional) gives a filesystem path to a certificate (PEM
  format).  privateKey (optional) gives a filesystem path to a a private key
  (PEM format).  caCertsDir (optional) gives a filesystem path to a directory
  containing trusted CA certificates to use to verify the server certificate.
  For example,
    <code>ssl:host=twistedmatrix.com:port=443:caCertsDir=/etc/ssl/certs</code>.
  </li>
  <li>UNIX.  Supported arguments: path, timeout, checkPID.  path gives a
  filesystem path to a listening UNIX domain socket server.  checkPID (optional)
  enables a check of the lock file Twisted-based UNIX domain socket servers use
  to prove they are still running.  For
  example, <code>unix:path=/var/run/web.sock</code>.
  </li>
</ul>

<h3>Servers</h3>

<ul>
  <li>TCP (IPv4).  Supported arguments: port, interface, backlog.  interface and
  backlog are optional.  interface is an IP address (belonging to the IPv4
  address family) to bind to.  For example,
  <code>tcp:port=80:interface=192.168.1.1</code>.
  </li>
  <li>TCP (IPv6).  All TCP arguments are supported, with interface taking an
  IPv6 address literal instead.  For example,
  <code>tcp6:port=80:interface=2001\:0DB8\:f00e\:eb00\:\:1</code>.
  </li>
  <li>SSL.  All TCP arguments are supported, plus: certKey, privateKey, and
  sslmethod.  certKey (optional, defaults to the value of privateKey) gives a
  filesystem path to a certificate (PEM format).  privateKey gives a filesystem
  path to a a private key (PEM format).  sslmethod indicates which SSL/TLS
  version to use (a value like TLSv1_METHOD). For example,
  <code>ssl:port=443:privateKey=/etc/ssl/server.pem:sslmethod=SSLv3_METHOD</code>.
  </li>
  <li>UNIX.  Supported arguments: address, mode, backlog, lockfile.  address
  gives a filesystem path to listen on with a UNIX domain socket server.  mode
  (optional) gives the filesystem permission/mode (in octal) to apply to that
  socket.  lockfile enables use of a separate lock file to prove the server is
  still running.  For example, <code>unix:address=/var/run/web.sock:lockfile=1</code>.
  </li>
  <li>systemd.  Supported arguments: domain, index.  domain indicates which
  socket domain the inherited file descriptor belongs to (eg INET, INET6).
  index indicates an offset into the array of file descriptors which have been
  inherited from systemd.  For
  example, <code>systemd:domain=INET6:index=3</code>.
  </li>
</ul>

</body>
</html>
